{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. 单神经元分类问题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "单神经元计算：\n",
    "\n",
    "第一步，计算 z , z=wX+b1\n",
    "\n",
    "第二步，通过激活函数计算a, a=σ(z)\n",
    "\n",
    "\n",
    "以MNIST手写数字识别问题为例\n",
    "\n",
    "MNIST 数据集来自美国国家标准与技术研究所, National Institute of Standards and Technology (NIST)\n",
    "\n",
    "数据集由来自 250 个不同人手写的数字构成\n",
    "\n",
    "其中 50% 是高中学生, 50% 来自人口普查局 (the Census Bureau) 的工作人员"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. 数据读取"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "利用网上的 MNIST 数据集(http://yann.lecun.com/exdb/mnist/)获取数据集压缩文件（切勿解压）：\n",
    "\n",
    "train-images-idx3-ubyte.gz --- 6万张28x28大小的训练数字图像\n",
    "\n",
    "train-labels-idx1-ubyte.gz --- 6万张训练图像的数字标记\n",
    "\n",
    "t10k-images-idx3-ubyte.gz  --- 1万张28x28大小的测试数字图像\n",
    "\n",
    "t10k-labels-idx1-ubyte.gz  --- 1万张测试图像的数字标记"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-1-f0ca348e51c4>:9: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "WARNING:tensorflow:From c:\\users\\dell\\appdata\\local\\programs\\python\\python37\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please write your own downloading logic.\n",
      "WARNING:tensorflow:From c:\\users\\dell\\appdata\\local\\programs\\python\\python37\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting ../data/train-images-idx3-ubyte.gz\n",
      "WARNING:tensorflow:From c:\\users\\dell\\appdata\\local\\programs\\python\\python37\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting ../data/train-labels-idx1-ubyte.gz\n",
      "WARNING:tensorflow:From c:\\users\\dell\\appdata\\local\\programs\\python\\python37\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:110: dense_to_one_hot (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.one_hot on tensors.\n",
      "Extracting ../data/t10k-images-idx3-ubyte.gz\n",
      "Extracting ../data/t10k-labels-idx1-ubyte.gz\n",
      "WARNING:tensorflow:From c:\\users\\dell\\appdata\\local\\programs\\python\\python37\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "训练集数量： 55000\n",
      "验证集数量： 5000\n",
      "测试集数量： 10000\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "# 载入数据集\n",
    "mnist = input_data.read_data_sets(\"../data/\", one_hot=True)\n",
    "\n",
    "# 显示数据集数量\n",
    "print(\"训练集数量：\", mnist.train.num_examples)\n",
    "print(\"验证集数量：\", mnist.validation.num_examples)\n",
    "print(\"测试集数量：\", mnist.test.num_examples)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练图像大小： (55000, 784)\n",
      "训练标签大小： (55000, 10)\n"
     ]
    }
   ],
   "source": [
    "# 显示数据集大小\n",
    "print(\"训练图像大小：\", mnist.train.images.shape)\n",
    "print(\"训练标签大小：\", mnist.train.labels.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADk5JREFUeJzt3W+IXfWdx/HPd237QNsH6iRpsHGnW8RMEDaNl7jgJmYpFrMUYkcqjVCzUDo+qH8CebA6JlREMSzbZH2wVCfb0KiJbSF/HxityOqksJRcRap1zFZkJs0mZO5gocYnRfPdB3NSpnHO71zvOfeem/m+XyBz7/meM+fLaT85987vnPMzdxeAeP6m7gYA1IPwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8I6nO93NnAwIAPDg72cpdAKJOTk5qZmbF21i0VfjO7TdKTki6T9F/uvj21/uDgoJrNZpldAkhoNBptr9vxx34zu0zSf0paL2mFpI1mtqLT3wegt8p8518t6T13f9/d/yzp55I2VNMWgG4rE/5rJP1hzvtT2bK/YmYjZtY0s2ar1SqxOwBVKhP++f6o8Kn7g919zN0b7t5YtGhRid0BqFKZ8J+StGzO+69IOl2uHQC9Uib8xyVdZ2ZfNbMvSPqupCPVtAWg2zoe6nP3j83sXkkvaXaob7e7/66yzgB0Valxfnd/QdILFfUCoIe4vBcIivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgSs3Sa2aTkj6U9Imkj929UUVT6J2pqalkfdeuXcn6448/nqybWW7N3ZPbDg0NJeuPPfZYsj48PJysR1cq/Jl/cveZCn4PgB7iYz8QVNnwu6RfmdnrZjZSRUMAeqPsx/6b3f20mS2W9LKZvevu43NXyP5RGJGka6+9tuTuAFSl1Jnf3U9nP6clHZS0ep51xty94e6NRYsWldkdgAp1HH4zu8LMvnThtaRvSnq7qsYAdFeZj/1LJB3MhnI+J2mfu79YSVcAuq7j8Lv7+5L+vsJe0KFWq5Vbe+KJJ5Lb7t27N1mfmUmP4qbG8dupp5w4cSJZ37JlS7K+du3a3NrAwEBHPS0kDPUBQRF+ICjCDwRF+IGgCD8QFOEHgqrirj50WdGtq9u2bcutFQ21Fd1WW7R90SXbZa7qLBpmnJycTNZTQ33vvPNOJy0tKJz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAoxvkvAYcPH07WU2PxZW6plaQVK1Yk66+++mqyXubW2WPHjiXrt9xyS7JedEtwdJz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAoxvn7wMTERLL+7rvvJuupe+qL7qcvGoffsWNHsr5169ZkfXR0NLdW9CyANWvWJOtFzyJIGRsbS9ZHRhb+1JOc+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gqMJxfjPbLelbkqbd/YZs2VWSfiFpUNKkpDvd/Y/da3NhGxoaStaPHz+erKfG6stORV00Hl5mvLxonP/AgQPJepnpwYeHh5PbRtDOmf9nkm67aNmDkl5x9+skvZK9B3AJKQy/u49L+uCixRsk7cle75F0e8V9AeiyTr/zL3H3M5KU/VxcXUsAeqHrf/AzsxEza5pZs9VqdXt3ANrUafjPmtlSScp+Tuet6O5j7t5w90aZSRsBVKvT8B+RtCl7vUlS+vGyAPpOYfjN7HlJ/yPpejM7ZWbfl7Rd0q1m9ntJt2bvAVxCCsf53X1jTukbFfeCHMuXL69t30XXCVx//fXJ+tVXX51b27lzZ3Lb7dvT55Si+/lTXzPLXv+wEHCFHxAU4QeCIvxAUIQfCIrwA0ERfiAoHt29AIyPj+fWih77XTTkVXS7cdE02DfddFNubXo698JQScW37C5enL6l5OjRo8l6dJz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAoxvkXgH379uXWih6tXXRbbNFYe9H2qbH8MrfkStJ9992XrK9atSpZj44zPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ExTj/Alc0Tl/n9mvXrk1uu2PHjmSdcfxyOPMDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCF4/xmtlvStyRNu/sN2bJHJP1AUitbbdTdX+hWk0i76667cmtTU1PJbWdmZpL1ouf+nzt3LllPefTRR5N1xvG7q50z/88k3TbP8p3uvjL7j+ADl5jC8Lv7uKQPetALgB4q853/XjP7rZntNrMrK+sIQE90Gv6fSPqapJWSzkj6cd6KZjZiZk0za7ZarbzVAPRYR+F397Pu/om7n5e0S9LqxLpj7t5w90bRAxkB9E5H4TezpXPeflvS29W0A6BX2hnqe17SOkkDZnZK0o8krTOzlZJc0qSke7rYI4AusKJnp1ep0Wh4s9ns2f5QXtE4/8MPP5ysHzp0KLdWNI5/9OjRZH1gYCBZj6jRaKjZbLb1EAau8AOCIvxAUIQfCIrwA0ERfiAowg8ExaO725S6NHkhX7m4fPnyZH3//v3J+vr163NrL774YnLb5557LlnfvHlzso40zvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBTj/Jnx8fFkfcuWLbm1orHwZ599tqOeFoLR0dHc2ksvvZTc9sSJE1W3gzk48wNBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUGHG+YumCrvnnvTUA0uWLMmtRR7H/+ijj5L11HHt5WPj8Wmc+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gqMJxfjNbJukZSV+WdF7SmLs/aWZXSfqFpEFJk5LudPc/dq/Vcg4ePJisF907vm7dugq7uXRMTEwk63fccUeynjquZumZpIuek4By2jnzfyxpi7sPSfoHST80sxWSHpT0irtfJ+mV7D2AS0Rh+N39jLu/kb3+UNKEpGskbZC0J1ttj6Tbu9UkgOp9pu/8ZjYo6euSfiNpibufkWb/gZC0uOrmAHRP2+E3sy9K2i9ps7v/6TNsN2JmTTNrFl1fD6B32gq/mX1es8Hf6+4HssVnzWxpVl8qaXq+bd19zN0b7t5YyBNaApeawvDb7J9kfyppwt13zCkdkbQpe71J0uHq2wPQLe3c0nuzpO9JesvM3syWjUraLumXZvZ9SSclfac7LVZjzZo1yXrR7aWvvfZabq1oKumhoaFk/cYbb0zWi0xNTeXWjh07ltz2wIEDyfqhQ4eS9aLjlhrOK5pi+4EHHkjWUU5h+N3915Ly/hf8RrXtAOgVrvADgiL8QFCEHwiK8ANBEX4gKMIPBBXm0d1FY+3Dw8PJemq8++67705uW3Tr6qpVq5L1IidPnsytzczMJLctM07fjq1bt+bW7r///lK/G+Vw5geCIvxAUIQfCIrwA0ERfiAowg8ERfiBoMKM8xd56qmnkvXUWHqz2Sy176Lti8baU2P1RdtefvnlyXrR9REPPfRQsl50/QTqw5kfCIrwA0ERfiAowg8ERfiBoAg/EBThB4JinD9TNJvQ0aNHc2vbtm0rte+nn346WS+aBntgYKDjfRc9G59pshcuzvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EJS18dz2ZZKekfRlSecljbn7k2b2iKQfSGplq466+wup39VoNLzsve8A8jUaDTWbzbYmW2jnIp+PJW1x9zfM7EuSXjezl7PaTnf/904bBVCfwvC7+xlJZ7LXH5rZhKRrut0YgO76TN/5zWxQ0tcl/SZbdK+Z/dbMdpvZlTnbjJhZ08yarVZrvlUA1KDt8JvZFyXtl7TZ3f8k6SeSviZppWY/Gfx4vu3cfczdG+7eKLp+HkDvtBV+M/u8ZoO/190PSJK7n3X3T9z9vKRdklZ3r00AVSsMv80+/vWnkibcfcec5UvnrPZtSW9X3x6Abmnnr/03S/qepLfM7M1s2aikjWa2UpJLmpR0T1c6BNAV7fy1/9eS5hs3TI7pA+hvXOEHBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IqvDR3ZXuzKwlaWrOogFJMz1r4LPp1976tS+J3jpVZW9/6+5tPS+vp+H/1M7Nmu7eqK2BhH7trV/7kuitU3X1xsd+ICjCDwRVd/jHat5/Sr/21q99SfTWqVp6q/U7P4D61H3mB1CTWsJvZreZ2Qkze8/MHqyjhzxmNmlmb5nZm2ZW65TC2TRo02b29pxlV5nZy2b2++znvNOk1dTbI2b2f9mxe9PM/rmm3paZ2X+b2YSZ/c7MHsiW13rsEn3Vctx6/rHfzC6T9L+SbpV0StJxSRvd/Z2eNpLDzCYlNdy99jFhM1sr6ZykZ9z9hmzZv0n6wN23Z/9wXunu/9onvT0i6VzdMzdnE8osnTuztKTbJf2Lajx2ib7uVA3HrY4z/2pJ77n7++7+Z0k/l7Shhj76nruPS/rgosUbJO3JXu/R7P95ei6nt77g7mfc/Y3s9YeSLswsXeuxS/RVizrCf42kP8x5f0r9NeW3S/qVmb1uZiN1NzOPJdm06RemT19ccz8XK5y5uZcumlm6b45dJzNeV62O8M83+08/DTnc7O6rJK2X9MPs4y3a09bMzb0yz8zSfaHTGa+rVkf4T0laNuf9VySdrqGPebn76ezntKSD6r/Zh89emCQ1+zldcz9/0U8zN883s7T64Nj104zXdYT/uKTrzOyrZvYFSd+VdKSGPj7FzK7I/hAjM7tC0jfVf7MPH5G0KXu9SdLhGnv5K/0yc3PezNKq+dj124zXtVzkkw1l/IekyyTtdvfHe97EPMzs7zR7tpdmJzHdV2dvZva8pHWavevrrKQfSTok6ZeSrpV0UtJ33L3nf3jL6W2dZj+6/mXm5gvfsXvc2z9KOibpLUnns8Wjmv1+XduxS/S1UTUcN67wA4LiCj8gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0H9P4TvLHTCfnYzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0. 0. 0. 1. 0. 0. 0. 0. 0. 0.]\n"
     ]
    }
   ],
   "source": [
    "# 可视化图像\n",
    "def plot_image(image):\n",
    "    plt.imshow(image.reshape(28, 28), cmap='binary')\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "# 可视化第二张训练图像\n",
    "plot_image(mnist.train.images[1])\n",
    "\n",
    "# 可视化第二张图像的标签\n",
    "print(mnist.train.labels[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. 构建模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\users\\dell\\appdata\\local\\programs\\python\\python37\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Colocations handled automatically by placer.\n"
     ]
    }
   ],
   "source": [
    "# 定义训练数据的占位符， x是784个像素点的特征值， y是10分类的标签值\n",
    "x = tf.placeholder(tf.float32, [None, 784], name=\"X\")\n",
    "y = tf.placeholder(tf.float32, [None, 10], name=\"Y\")\n",
    "\n",
    "# 初始化w为shape=（784,10），服从标准差为0.01的随机正态分布的数\n",
    "W = tf.Variable(tf.random_normal([784, 10], stddev=0.01), name=\"W\")\n",
    "# 初始化b为0\n",
    "b = tf.Variable(tf.zeros([10]), name=\"b\")\n",
    "\n",
    "# 定义向前传播计算\n",
    "forward = tf.matmul(x, W) + b\n",
    "\n",
    "# 定义分类结果softmax分类\n",
    "pred = tf.nn.softmax(forward)\n",
    "\n",
    "# 定义损失函数\n",
    "with tf.name_scope(\"LossFunction\"):\n",
    "    loss_function = tf.reduce_mean(-tf.reduce_sum(y * tf.log(pred) + (1 - y) * tf.log(1 - pred), reduction_indices=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. 训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\users\\dell\\appdata\\local\\programs\\python\\python37\\lib\\site-packages\\tensorflow\\python\\ops\\math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.cast instead.\n",
      "Train Epoch 01 Loss= 1.090827823 Accuracy= 0.8536\n",
      "Train Epoch 02 Loss= 0.842468679 Accuracy= 0.8754\n",
      "Train Epoch 03 Loss= 0.744230688 Accuracy= 0.8834\n",
      "Train Epoch 04 Loss= 0.687984586 Accuracy= 0.8904\n",
      "Train Epoch 05 Loss= 0.651327908 Accuracy= 0.8950\n",
      "Train Epoch 06 Loss= 0.624509335 Accuracy= 0.8986\n",
      "Train Epoch 07 Loss= 0.604190111 Accuracy= 0.9008\n",
      "Train Epoch 08 Loss= 0.589582622 Accuracy= 0.9022\n",
      "Train Epoch 09 Loss= 0.575418770 Accuracy= 0.9058\n",
      "Train Epoch 10 Loss= 0.564585030 Accuracy= 0.9072\n",
      "Train Epoch 11 Loss= 0.555016398 Accuracy= 0.9092\n",
      "Train Epoch 12 Loss= 0.547104895 Accuracy= 0.9106\n",
      "Train Epoch 13 Loss= 0.540051103 Accuracy= 0.9122\n",
      "Train Epoch 14 Loss= 0.533957124 Accuracy= 0.9122\n",
      "Train Epoch 15 Loss= 0.527852356 Accuracy= 0.9132\n",
      "Train Epoch 16 Loss= 0.522758484 Accuracy= 0.9142\n",
      "Train Epoch 17 Loss= 0.518281937 Accuracy= 0.9138\n",
      "Train Epoch 18 Loss= 0.514368415 Accuracy= 0.9150\n",
      "Train Epoch 19 Loss= 0.510011733 Accuracy= 0.9162\n",
      "Train Epoch 20 Loss= 0.507149398 Accuracy= 0.9170\n",
      "Train Epoch 21 Loss= 0.503677487 Accuracy= 0.9178\n",
      "Train Epoch 22 Loss= 0.500926316 Accuracy= 0.9166\n",
      "Train Epoch 23 Loss= 0.499000430 Accuracy= 0.9170\n",
      "Train Epoch 24 Loss= 0.495698571 Accuracy= 0.9186\n",
      "Train Epoch 25 Loss= 0.493197560 Accuracy= 0.9212\n",
      "Train Epoch 26 Loss= 0.491086245 Accuracy= 0.9204\n",
      "Train Epoch 27 Loss= 0.489388674 Accuracy= 0.9208\n",
      "Train Epoch 28 Loss= 0.486955225 Accuracy= 0.9216\n",
      "Train Epoch 29 Loss= 0.484734178 Accuracy= 0.9216\n",
      "Train Epoch 30 Loss= 0.483102679 Accuracy= 0.9210\n",
      "Train Epoch 31 Loss= 0.481649280 Accuracy= 0.9218\n",
      "Train Epoch 32 Loss= 0.480098337 Accuracy= 0.9220\n",
      "Train Epoch 33 Loss= 0.478767246 Accuracy= 0.9230\n",
      "Train Epoch 34 Loss= 0.476945937 Accuracy= 0.9224\n",
      "Train Epoch 35 Loss= 0.475882530 Accuracy= 0.9232\n",
      "Train Epoch 36 Loss= 0.474474937 Accuracy= 0.9238\n",
      "Train Epoch 37 Loss= 0.473806441 Accuracy= 0.9228\n",
      "Train Epoch 38 Loss= 0.471931845 Accuracy= 0.9234\n",
      "Train Epoch 39 Loss= 0.470839739 Accuracy= 0.9226\n",
      "Train Epoch 40 Loss= 0.469887793 Accuracy= 0.9234\n",
      "Train Epoch 41 Loss= 0.468856692 Accuracy= 0.9242\n",
      "Train Epoch 42 Loss= 0.468088090 Accuracy= 0.9244\n",
      "Train Epoch 43 Loss= 0.466547161 Accuracy= 0.9250\n",
      "Train Epoch 44 Loss= 0.466059327 Accuracy= 0.9250\n",
      "Train Epoch 45 Loss= 0.464698195 Accuracy= 0.9240\n",
      "Train Epoch 46 Loss= 0.464338869 Accuracy= 0.9236\n",
      "Train Epoch 47 Loss= 0.463424206 Accuracy= 0.9240\n",
      "Train Epoch 48 Loss= 0.462737888 Accuracy= 0.9246\n",
      "Train Epoch 49 Loss= 0.461778700 Accuracy= 0.9248\n",
      "Train Epoch 50 Loss= 0.461308300 Accuracy= 0.9254\n"
     ]
    }
   ],
   "source": [
    "train_epochs = 50  # 迭代次数\n",
    "learning_rate = 0.01  # 学习率\n",
    "\n",
    "# 梯度下降优化器 设置学习率和优化目标损失最小化\n",
    "optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss_function)\n",
    "\n",
    "# 定义预测类别匹配情况\n",
    "correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))\n",
    "\n",
    "# 定义准确率，将布尔值转化成浮点数，再求平均值\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "\n",
    "sess = tf.Session()  # 建立会话\n",
    "init = tf.global_variables_initializer()  # 变量初始化\n",
    "sess.run(init)\n",
    "\n",
    "# 每个批次的大小，每次放入的大小，每次放入 100张图片 以矩阵的方式\n",
    "batch_size = 100\n",
    "\n",
    "# 计算一共有多少个批次，数量整除大小训练出有多少批次\n",
    "n_batch = mnist.train.num_examples // batch_size\n",
    "\n",
    "# 批次迭代训练\n",
    "for epoch in range(train_epochs):\n",
    "    for batch in range(n_batch):\n",
    "        xs, ys = mnist.train.next_batch(batch_size)\n",
    "        sess.run(optimizer, feed_dict={x: xs, y: ys})\n",
    "    # 批次训练完成之后，使用验证数据计算误差与准确率\n",
    "    loss, acc = sess.run([loss_function, accuracy], feed_dict={x: mnist.validation.images, y: mnist.validation.labels})\n",
    "    # 显示训练信息\n",
    "    print(\"Train Epoch\", '%02d' % (epoch + 1), \"Loss=\", '{:.9f}'.format(loss), \"Accuracy=\", \"{:.4f}\".format(acc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. 评估模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test Accuracy =  0.9228\n",
      "Validation Accuracy =  0.9254\n",
      "Train Accuracy =  0.9202\n"
     ]
    }
   ],
   "source": [
    "# 测试集上评估模型预测的准确率\n",
    "accu_test = sess.run(accuracy, feed_dict={x: mnist.test.images, y: mnist.test.labels})\n",
    "print(\"Test Accuracy = \", accu_test)\n",
    "\n",
    "# 验证集上评估模型预测的准确率\n",
    "accu_validation = sess.run(accuracy, feed_dict={x: mnist.validation.images, y: mnist.validation.labels})\n",
    "print(\"Validation Accuracy = \", accu_validation)\n",
    "\n",
    "# 训练集上评估模型预测的准确率\n",
    "accu_train = sess.run(accuracy, feed_dict={x: mnist.train.images, y: mnist.train.labels})\n",
    "print(\"Train Accuracy = \", accu_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. 模型预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[6 0 5 4 9 9 2 1 9 4]\n"
     ]
    }
   ],
   "source": [
    "# 转换pred预测结果独热编码格式为数字0-9\n",
    "prediction_result = sess.run(tf.argmax(pred, 1), feed_dict={x: mnist.test.images})\n",
    "\n",
    "# 查看第100-109张测试图片的预测结果\n",
    "print(prediction_result[100:110])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkQAAAEHCAYAAABPxFkFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XeYVEXaNvD7kSSgBBlUJDgEXUVE0qKoKIoCAiK8C6Liiqu+8OlrwkXdVXEJZqIYUMQAAi6C6CWIEkSiiYwCAisiCKggSjAsqb4/zpmyquzu6Th9ps/9uy4unpo6obqrT0/NqTpVopQCERERUZgdle0CEBEREWUbG0REREQUemwQERERUeixQUREREShxwYRERERhR4bRERERBR6hTaIRGR/Ifn5IvJ5IicVkVdEpGsi+0Q5Ti0RmSUi60RkrYjkp3rMOM6pyy4iY0SkfoxtW4nIuWk6b2kRGS0iG0TkCxH5S5LHCXJ99hSRjf6/nqkeL85zzhORZn48Q0Qqxdi2c6z6TvC87URkvYj8R0T+keQxglyXh0Vkpf/v7VSPF+c5s3VtzvPrsuD1Hp/kcQJbn/6xKojINhF5Oh3Hi+N8Wbk2/eOVEJEVIjI9yf0DW5ci8riIfO7/657q8eI8Z7auze4islpE1ojIE4VtXzIdJ82icQAeVkrNFpFjABxJ5iAiUlIpdSjR/ZRSNxWySSsA+wF8mEy5HPcD+F4pdaqIHAXguDQcMzBE5DgA/wLQDIACsExE3lZK/ZjEsZKtz/aFbNIZwHQAaxM9tklESgB4BsClAL4BsMR/rSkdN2B+VUo1SvUgxeTaBIAeSqmlaTpWUA0CMD+VAwT92jTcAWAdgAppOl4giEgHAE0ANAJQBsB8EXlXKbU3iWMF+toUkSoABgNoqpTaKSJjRaS1Uur9aPvE3WUmIseIyPsislxEPhORK4zskv7JVovIFBEp5+/TVETmi8gyEZkpItWSfnV/LE99ACWVUrMBQCm1Xyn1S4Tt5onICBH50G8RN/d/3t+/4zILwDj/L4LBIrLEfx29/e1ERJ4W7w7UOwCOd45d8BdMO/+9WeW/T/kA/h+APv5fjS1TfMk3AHjUf61HlFK7UjlY0OoTQFsAs5VSu/1G0GwA7SKUe7P/F86n/r96/s9fEZFhIvIBgMdFpLyIvOTX54qC1yciZUXk3/5rmwSgrHPsPD++zt9mlYi86v/F0gnAYL8+66bwWpsD+I9SapNS6gCAfwO4opB9ogpgXcZb7ly5NtMqiPUpIk0BnABgVoxtcuHahIjUANABwJhUjuMfK2h1WR/AfKXUIaXUzwBWIfL3bC5cm3UAbFBK7fTTcwDE7llRSsX8B2C//39JABX8OA/AfwAIgHx4f9Gf5+e9BKAvgFLwWnhV/Z93B/CSH78CoGuEc90NYGWEfyMjbFvwF8FUACvgtQRLRNhuHoAX/PgCAJ/7cX8AywCU9dO9ADzgx2UALAVQG8D/wPvlXALASQB+Kii7f+xmAKoC2Aqgtv/z44xz9I3yvl4U5bV+GGHbSv7xhwFYDmAygBMKq7tiVp99C95/P90v0nsHYDOA+/34OgDTjTJML/gMAHgEwLXG+7cBQHkAdxnlbgjgEIBmxrHzAJwBYD2APKc+I75OP69HlNc6JcK2XQGMMdJ/BfB0rtSlv/0heNfQxwA6R9lmHor5tWmc6zN/m34AJMeuzaP811gTwPWI8llFDlyb/vZTADSFd6dieo7VZRsAiwGU88uzCcDfc/HaBFAZ3h34fL8e3gAwLVa9JdJlJgAeEZEL4HVNVYf3FwMAbFVKLfbj8QBuB/AegAYAZosI/DdmR6wTKKUGw2vYxKMkgJYAGgPYAmASvIv1xQjbvuYff4F4/eAFfdFvK6V+9eM2ABrK7320FQGcAu/D8JpS6jCA7SIyN8LxzwGwQCn1lX+e3YUVXin1AbzblvEoCaAGgMVKqbtE5C4AQ+D9Ik1W0OpTIh0iyravGf8PN34+2a8nwKvPTiLS108fDaAWvPoc6ZdvtYisjnD8i+F9We7yt4unPicAmFDYdr5EXmu8xwtSXQJALaXUdhGpA2CuiHymlPoywnbF/doEvO6ybSJyLLwv3b/C685PVtDq8xYAM5RSW/3jx1Ksr00R6QhvaMIyEWkVzz6FHRIBqkul1CwR+TO8RtdOAB/Ba3hGUqyvTaXUjyJyM7y2wRF4r7lOrH0SaRD1gNeia6qUOigim+F9kIE/fpkreB+ENUqpFvGeQETu9s/jWqCUut352TcAViilNvn7vgXvDY7UIIpUPgD42Tw9gNuUUjOdMrWPsP8fih7HNvYOIhfB/sIo8ItSyh1Q9gOAXwC86acnA7gxkfNFEMT6bGWka8D7SyISFSV26/MvSqn1TpncfSJJpj57wPtLzfUfpZQ7EPIbeH9tF6gBYHsi53MErS6hlNru/79JRObB+8MlUoOouF+bUEpt8//fJyIT4XWJptIgClp9tgDQUkRuAXAMgNIisl8pFelhgOJ+bZ4Hr7HWHt57XkFExiulrk3knIag1SWUUg8DeNjfdyKAjVEOnQvX5jQA0/x9ewE47G5jSuSx+4rwWs4H/UKdbOTVEpGCCrwawCJ4tzWrFvxcREqJyBmxTqCUGqyUahTh3x8qFcASAJVFpKqfvhj+gDoReVREuhjbdvd/fj6APUqpPRGONxPAzSJSyt/2VBEpD2ABgKv8vtJq8G7ZuT4CcKGI1Pb3LRjwvA/AsVFe6wdRXmukSlXwKrWV/6PWSH3wYNDqcyaANiJSWUQqw/vLY6Z/rnEFfdi+7sb/H0U5/UwAt4n/LSsijf2fL4D/5SEiDeDdmne9D+BK8QblxVufE6K81khPhSwBcIqI1BaR0gCuApDKk1iBqku/Dsv4cR68XzI5eW2KSEn5fWxLKQAdAST09FAEgapPpVQPpVQtpVQ+vG6dcQWNoVy7NpVS/1RK1fBf61UA5qbQGAICVpf+tVLw3jWE9x7P8tM5dW36xzze/78yvDudMceFJXKHaAKAaSKyFF6f3RdG3joAPUXkeXitzVFKqQP+bbSRIlLRP9cIAGsSOGdUSqnD4t1yfd+/sJYBeMHPPhP2L5gfReRDeE8M3BDlkGPg9TUu94+3E944pTfhNbY+g9fX/YenLJQ3gr0XgKniPQH2PbwniKYBmCLeQLrblFILU3jJ9wJ4VURG+GX7WwrHAoJXn7tFZBC8xgIADFS/30JtCPu2cRkR+QReg/7qKIcc5JdvtV+fm+H9shoF4GXxbsevBPBphLKsEZGH4T2BcRjeGLXr4Q1+fkFEbofXHx7pjkc8r/WQiNwK78ukBLwxAqm8j4GqSwCnA3heRI7Aq6PH1O9P0OXatVkGwEz/F0IJeAM3X4i9S6GCVp+x5NS1mQFBq8tSABb6bdG98MZyFXSZ5dq1CQBPishZfjxQKbUh1sbi3XzILSIyUynV1o/nwRugleuPxOYkEakA4EWlVDc/vRneQMuUnrKj7OC1mTt4beYWXpvFfx6iiAoqlYo/5c2P0S3b5aD04LWZO3ht5hZemzl6h4iIiIgoEVzLjIiIiEKPDSIiIiIKPTaIiIiIKPQSGlSdl5en8vPzM1QUiseyZct2KaWqFr5l4Vif2bV582bs2rWr0Kl/48G6zK501iXA+sw2Xpu5I5G6TKhBlJ+fj6VLQ/UUXuCIyNfpOhbrM7uaNWuWtmOxLrMrnXUJsD6zjddm7kikLtllRkRERKHHBhERERGFHhtEREREFHpsEBEREVHo5eTSHQMGDNDx2LFjdfz6669b26V7ICQREREVT7xDRERERKHHBhERERGFXk50mc2bN89KP//88zouX768jt25INhlRkRERADvEBERERGxQURERETEBhERERGFXrEdQ7Rv3z4dd+vWzcq77rrrdPz444/rWCRtay8SEVGavfXWW1a6T58+Op44caKV16JFiyIpE4UH7xARERFR6LFBRERERKFXbLvMRo0apeOjjz7ayuvbt6+OS5Ysti+RItiwYYOOb7zxRivvq6++0vH48eOtvN27d+v4nXfesfIee+wxHVetWjUt5SSixE2dOtVKf/311zp2r/e1a9cWSZkoPHiHiIiIiEKPDSIiIiIKPTaIiIiIKPSK7QAb83H63r17W3nVqlUr6uJQBq1bt07Hl112mY63bNlibaeU0nHbtm2tvFNOOUXH7tiDZcuW6XjOnDlWXl5eXhIlpgLfffedla5SpYqOzfF9//3vf63tvv/++6jHnD9/vo6XL18ed1leeOEFHZ900klRt3viiSes9BVXXBH3OSg1S5YsyXYRcsrevXut9IMPPqjjJ598MqljXn755Tp+6qmnrLyTTz45qWMGBe8QERERUeixQUREREShV2y6zMyZqQHgwIEDOj799NOLujiUQffee6+Vnjt3ro7NbrLq1atb25ndMatXr7bybrrpJh2PHj3ayjO3bdOmjZW3ePFiHZctW7bQsoeF+R663YymDz74wEqbswub02X88MMPMfczmV2jicw+b+63cePGqNstWLDASudil9nkyZOt9Guvvabjhx56yMqrX79+kZSpMNdcc022i1AsLFq0SMe9evWy8r744gsdx7p2zj33XB2vX7/eyps+fbqOP/74Yytv06ZNOj7mmGPiLHFw8A4RERERhR4bRERERBR6bBARERFR6BWbMUTvvfde1Lx27doVYUkoE/75z3/qeMSIEVbewYMHddy9e3cdm4+QAsDIkSN17D5W3aNHDx27Y4HuvPNOHa9YscLK+/TTT3V84YUXRn8BOe7999+30vfff7+Od+3aZeXFGptgjj+INRaodevWOjbHhhW2n/n5cKffMB/l79y5s5VXs2ZNHf/tb3+LWv5c4T4ubY47uf766628TI8hMqdm2LNnT9TtypQpk9FyFFcLFy600h07dtSxO/b2hBNO0PHw4cOtvLp16+q4cePGOv7ss8+s7czvXXcZJPP6vuqqqwote9DwDhERERGFHhtEREREFHrFpsvMXN0eAEqXLq1jrlBe/Lz00ktW2lxx3u0GOeuss3RsPhJs3uIF/vgZicbtEjnxxBN17M7e2qVLFx2bt4MB+9HUXLR9+3Ydm9MWAPZj8q1atbLyduzYoeMNGzZYeU2bNtVxv379dNykSRNru+OPP17H5rWeikOHDun4rbfesvLOO+88HbtddLnInLkdsLtdXn75ZSuvU6dOGS2LOZvyr7/+auWZ3aPmZyLs9u/fr+Nbb73VyjO7yZo3b27ljR8/Xsf16tWL61xm9xlgf8+a1zMA3HDDDTp2v5///Oc/x3W+bOIdIiIiIgo9NoiIiIgo9NggIiIiotAL9Bgis/949+7dVt4ll1yS0XPPmzdPx5MmTYq6XaVKlax0y5YtdWyuzA4ktsxALjIf37zxxhujbmfWO2AvM+D2SyfDHZNiLs2wbds2K89cQmLAgAFW3syZM1MuS5CsXbvWSp9xxhlRtzXHDbnLbCxbtkzH5hgsAHj77bd17D4Wn2klS/7+dZfpcTHFjfndZE5zAQBHjhzR8VFHpf9v6FiP3ZvlMh8nDztzzKX7WHyJEiV0fN9991l58Y4biqVGjRo6dscIDh06VMfmOKfigneIiIiIKPTYICIiIqLQC3SXmfn47qpVq6w8d0X0ZBw4cEDH//jHP6w8cxbPk08+2co79thjdWzOcAsAzz77rI7dFaXdldRz3c6dO6305ZdfrmO3+9DsJjMfxwaA2rVrZ6B0kf3v//6vlV69erWO3dnSzVvCxXFlZ9eYMWOsdKwuXvPavPLKK628Dh066Njtbv7vf/+bShGpCKxcudJK//zzzzo2v/soe6ZOnRo1r1mzZjrOdNdwxYoVrfTAgQMzer5M4x0iIiIiCj02iIiIiCj02CAiIiKi0Av0GKJYklmuw3x8FLDHi4wbN87KM6cnd5d6iLXqsrkkQO/eva08s2/e7XvNRT/99JOV3rJlS9RtzRW2zZXvAfsx0kwrVaqUle7atauOx44da+Vt2rRJxw0bNsxswYqAuzr1iBEjom5rPm67fv16K2/KlClR9zOXSenZs6eO3Wk0zNXuKf0mTpwYNe/SSy+10pkeNxSrLBRZrO9ScwwfJYZ3iIiIiCj02CAiIiKi0At0l9nXX38dNS+ZlXPdVYFnzZql49mzZ1t55i37RGaYbtu2rY5/++03K898fDUMXWZuV0os7grbQWF2j7p1lmurb7srzse6LR+L+Uj+oEGDrLzp06fr2Jxt1+2eu+2223TsdgFceOGFSZUrbJYsWWKlzUe13e+mRo0a6fiOO+7IbMEcnIqBgoJ3iIiIiCj02CAiIiKi0At0l9n333+f8jG+/fZbHZsLSwLAhAkTdHzxxRenfC4AKFu2rI7dhUgXLlyo4+7du6flfEH2xhtvZLsIKVu6dKmOY3WB5gJz8VPAXsQxEeZ+7jVnLvxqdp89/vjj1naDBw+OGAPAgw8+qGN3xvpy5colUeLiy12I1XzC74YbbrDy3M+vqUGDBjp2n6L95ZdfdLx9+3YrLx2LhZrchZ0psquvvlrHL774opVnpitUqGDlmd3i559/vpW3fPlyHS9atEjHX3zxhbWdufB5Iq677jod/+lPf7Ly3EWgs4V3iIiIiCj02CAiIiKi0GODiIiIiEIv0GOISpcuHTVv69atOo71CPv48eN1/N1331l55513XgqlS9y+ffuK9HzZtm3bNittjg+oXr16URcnKeeee66ODx8+bOUdOnSoqItT7DVt2jRi3KZNG2u7IUOG6HjGjBlWnrmi9ueff27lvfLKKzoOw8rsd911l5V+5plnom5rTh/ijtUxx1O6477y8vJ0vHv3bivPnHk8WeY4T3eKE3MG80qVKqV8rlxhXh+LFy+28swxP3feeaeVZ44Pu+aaa6w8c1qGPXv2pKWcpvvuu0/H7ve/+bs4m9OZ8A4RERERhR4bRERERBR6ge4yMx8LdG/NPv/88zp+6qmnoh7jnHPO0bHbxTF//nwdu7fsk2Wew+0iC8MtX/OxavfxTPN2+LPPPltURUrI2rVrrXSnTp107N7mdR8dpeS1aNHCSptTNtxzzz1WnvkY/ptvvmnlXXHFFTo2H/PNVXPmzElqv/z8fCttdlvVr1/fyvvkk0+iHsdcwDmRGf1NZvede4xjjjlGx0W5yHPQmcNEhg0bZuU9/PDDOj7ttNPiPqa5kHUsvXr10vFRR8V/T8WcDuC5556z8vbu3atjdpkRERERZREbRERERBR6bBARERFR6AV6DJH52Ky7jMDkyZN1PHz4cCvPXILguOOO07HbP52Jx6ZHjhypY7NfHgBat26d9vMFjfmexnp/Y02pkE3Tpk2z0jt37tTxBRdcUNTFIQD9+vWz0nXq1NHxLbfcYuWZ3wXucgC5+Bj+2LFjrXSpUqV0fMIJJ1h55iPX7vIcBw4c0LE5bgewx0Kay60AwKmnnqpjc9X6WI9tT5w40Uq//vrrUbelwrVr185Km+NhExnjk2mdO3eOmvfCCy/o2F3GpygF590iIiIiyhI2iIiIiCj0At1lZurbt6+VNmfZ7NOnj5VnPoZvPkJ60003WduZaXdl6KOPPjpqWVq2bKnjLVu2WHnmbJzvvvuulVe5cuWox8wV5m1695a9OVN4kGbtNmd2HTp0aNTt3M8gFQ23q8vs0nFnXF65cqWOP/jgAyvPnEIhVzRv3jzj56hSpYqOe/bsmfLxfvjhByvNLrP0ClI3mcm9Vk1HjhwpwpJEF8x3joiIiKgIsUFEREREoccGEREREYVesRlD1L17dyttrsw7evRoK69cuXI6NleDNh+JB4D27dvreNeuXVae2d9pPk7qnnvVqlVWnrnysLmad1iYSwJMmjTJymvVqpWO3frs1q1bJotlMccMAfZnxHzMHgAuu+wyHTdo0CCzBaOItm/fbqUHDRqk41jLRTRp0iRjZaLkxRpL4ubF2paCbcaMGVZ6x44dUbctirFw8eAdIiIiIgo9NoiIiIgo9IpNl5lr/PjxOjYfdQfs1X/Nbht3psyaNWtGPf5bb72l40WLFll5l1xyiY5HjBhh5TVq1ChWsUOlbt26VtqcAmHNmjVW3pVXXqnjCRMmWHnm7LvpMGDAACttTo9QtWpVK8/8/JQvXz6t5aDo3n//fR2702ps2LBBx26XmTnjbTZXzaboYnVzunmxtqVg27hxo5WO1f3ZuHHjTBcnLrxDRERERKHHBhERERGFHhtEREREFHrFdgyROa5k8ODBVl7Xrl11bI4BWbhwobXdunXrdGw+Eg4AF198sY779+9v5V100UU6Duo06UFQvXp1K927d28d33HHHVbe5MmTdXzppZdaeX/96191HGtJFdOyZcus9DPPPKNjc9oEwB5rcv/991t5ubhCehC5q6jfdtttOv7666+j7terVy8rbX5WSpcunabSUTrVqlXLSptTdcSqaypeZs6cGTXvggsusNLmZyCb+NuciIiIQo8NIiIiIgq9YttlFsvZZ58dMabsMmen3rJli5U3ZMgQHZtdawDwyCOP6PjMM8/UsTsT8cSJE3Xszji9Z88eHZ9wwglWnjmjalAe/wwac2oCwO7WLFky/q8Rc9bpjh076njFihVR93G7LXv06KHjUaNGxX1uCgZz2hLAHp7w8ssvF3VxKAvc7uxEvkMyiXeIiIiIKPTYICIiIqLQY4OIiIiIQi8YHXcUCubj7ffee6+Vd/DgQR1PmTLFytu2bZuOzcdy3Ue1zanh3T5qc1qF//u//7PyOG6ocAMHDrTSn3zyiY7btm1r5ZmrWpvTXgDAO++8o+Off/5Zx+4SDeZYMXPKBAA4//zz4y02FQMtW7bU8dy5c608c/wg5Y5Dhw5Z6SNHjug4m1PZ8A4RERERhR4bRERERBR67DKjrMjLy7PSI0aMiBgDwIIFC3T8yy+/6Nh9RNfsFmvUqJGV16JFi6TLSsCf/vQnK212oQ0YMMDKi3eF8rJly+rYnWrBPD5nC89t119/fcSYircTTzwxat68efOs9KZNm3Rcr169TBWpULxDRERERKHHBhERERGFHhtEREREFHocQ0SB566MXKBdu3ZFXJLwevbZZ620uTr1tGnTrDxzGQ537FH79u113LdvXx1Xq1YtHcUkooAYPny4ld68ebOO69ata+XVrFmzKIpUKN4hIiIiotBjg4iIiIhCj11mRFSocuXKWen+/ftHjImIAKBixYpW2p2FPIh4h4iIiIhCjw0iIiIiCj02iIiIiCj02CAiIiKi0GODiIiIiEKPDSIiIiIKPVFKxb+xyE4AX2euOBSHk5VSVdNxINZn1rEuc0fa6hJgfQYAr83cEXddJtQgIiIiIspF7DIjIiKi0GODiIiIiEKPDSIiIiIKPTaIiIiIKPTYICIiIqLQY4OIiIiIQo8NIiIiIgo9NoiIiIgo9NggIiIiotBjg4iIiIhCjw0iIiIiCj02iIiIiCj02CAiIiKi0GODiIiIiEKPDSIiIiIKPTaIiIiIKPTYICIiIqLQY4OIiIiIQo8NIiIiIgo9NoiIiIgo9NggIiIiotBjg4iIiIhCjw0iIiIiCr1CG0Qisr+Q/HwR+TyRk4rIKyLSNZF9ohzncRH53P/XPdXjxXlOXXYRGSMi9WNs20pEzk3TebuLyGoRWSMiT6RwnEDWp4g0EpGP/Ne3OpfrU0TKicg7IvKF/3ofS/I4gaxL/zjvichPIjI91WMlcM5sXZsPi8jWwuojjuMEuT75XZvYMQJbl/6xKojINhF5Oh3Hi+N880SkmR/PEJFKMbbtHKuukzh3CRFZEc93UbG9QyQiHQA0AdAIwNkA7haRCkkeq2Qy+ymlblJKrY2xSSsA6fgFWgXAYACtlVJnADhBRFqnetyA+QXAdf7rawdgRKyLJpag16dviFLqNACNAZwnIpel6bhBMRjAX1M9SDGpy2kAmqfpWIHD79qc+64FgEEA5qdygBTqsr1S6qcYm3QGkLYGEYA7AKyLZ8O4G0QicoyIvC8iy0XkMxG5wsguKSJj/Vb1FBEp5+/TVETmi8gyEZkpItUSex0x1QcwXyl1SCn1M4BV8H6RuuWeJyIjRORD/6+b5v7P+4vIaBGZBWCc34ocLCJL/NfR299ORORpEVkrIu8AON45dkGrt53/3qzy36d8AP8PQB8RWSkiLVN4rXUAbFBK7fTTcwD8JYXjBa4+lVIblFIb/Xg7gO8BVI1Q7mJfn0qpX5RSH/jxAQDLAdRI9nhBq0sAUEq9D2BfIeUu9nXpv9aPlVI7UjmGKYD1ye/aJAWwLiEiTQGcAGBWjG02i3dX8FP/Xz3/56+IyDAR+QDA4yJSXkRe8utyRcHrE5GyIvJv/7VNAlDWOXaeH1/nb7NKRF4V7y5fJwCD/bqsm+JrrQGgA4Axce2glIr5D8B+//+SACr4cR6A/wAQAPkAFIDz/LyXAPQFUArAhwCq+j/vDuAlP34FQNcI57obwMoI/0ZG2LYNgMUAyvnl2QTg7xG2mwfgBT++AMDnftwfwDIAZf10LwAP+HEZAEsB1AbwPwBmAygB4CQAPxWU3T92M3i/uLcCqO3//DjjHH2jvK8XRXmtH0bYtjKAb/z3uiSANwBMK6zuilN9Ovs1h9eiPyoX69PZrxK8z26dXKtLeH+1T4+Rn2t1uT+ZazLo9Ql+1yb8XRvgujzKfy9rArgewNNRyr8ZwP1+fB3869gvw3QAJfz0IwCu9eNKADYAKA/gLqPcDQEcAtDMOHYegDMArAeQ59RlxNfp5/WI8lqnRNl+CoCmKOS7qOBfIre8BMAjInIBgCMAqsNrZQLAVqXUYj8eD+B2AO8BaABgtogA3oc85l9RSqnB8G5XFkopNUtE/gzvw7MTwEfw3vRIXvP3WSBe32lBV8zbSqlf/bgNgIbyex9tRQCnwLuwX1NKHQawXUTmRjj+OQAWKKW+8s+zO47yfwDvFnShlFI/isjNACbBe+8/hPeXTCoCVZ+6UN5fQ68C6KmUOhJls2JdnwXEu+X8Grwvrk2J7OseCgGsyzjlRF2mWaDqk9+1KX3XBqouAdwCYIZSaqt//FheM/4fbvw1AyI9AAASJUlEQVR8sl9HgFeXnUSkr58+GkAteHU50i/fahFZHeH4F8NryOzyt4unLicAmFDYdgAgIh0BfK+UWiYireLZJ5EGUQ94rfOmSqmDIrIZ3osHvJauScH7IKxRSrWI9wQicrd/HtcCpdTt7g+VUg8DeNjfdyKAjVEOHal8APCzeXoAtymlZjplah9h/z8UPY5t7B1ELoL9ISvwi1LqD33hSqlp8MYqQER6ATjsbpOgwNWneOMS3oH31+PHMQ5d7OvTNxrARqXUiETOFUHg6jIBuVKX6RS4+uR3bdKCVpctALQUkVsAHAOgtIjsV0r9I8L+Kkrs1uVflFLrnTK5+0Qsehzb2DuI9IB3R8z1H6WUO+D8PHiNtfbw3vMKIjJeKXVttOMnMqi6IrzW1kH/A3aykVdLRAoq8GoAi+DdCqta8HMRKSUiZ8Q6gVJqsFKqUYR/kX55lhBvABxEpCG823Kz/PSjItLF2Ly7//PzAexRSu2JcPqZAG4WkVL+tqeKSHkACwBc5Z+vGrzbr66PAFwoIrX9fY/zf74PwLFRXusHUV5rxC9cETne/78yvFZ+fH2i0QWtPksDeBPAOKXUZCcvF+vzIXh1cGek/AQFqi5jycW6zIBA1Se/a1P6rg1UXSqleiilaiml8uF10Y0raAyJyDjxx335uhv/fxTl9DMB3CZ+C0hEGvs/XwC/kSYiDeB9ZlzvA7jS+GzFU5cTorzWPzx9p5T6p1Kqhv9arwIwN1ZjCEisQTQBQDMRWQrvhX5h5K0D0NO/LXYcgFHKGyzaFd7Aq1Xw+vnS+YVSCsBCEVkL7y/ta5VSBbdxzwTwrbHtjyLyIYDnANwY5XhjAKwFsFy8xyGfh3cH7U14fw19BmAUIozMV94AvF4ApvqvdZKfNQ1AF0nDwE0AT/qvdTGAx5RSG1I8XtDq80p4t1mv99+vlSJScJs7p+pTvIF+98MbrLrcP95NyR4PwatLiMhCAJMBtBaRb0SkrZ+VU3UJACLyhIh8A6Cc/1r7p3I8BK8++V2bvKDVZSwNYXfPlRGRT+A9pdUnyj6D4H0+Vvt1Ocj/+SgAx/iv7R4An7o7KqXWwLvrON9/rcP8rH/De5JxhaQ4qDpRolRCd6yKBRGZqZRq68fz4A22W5rdUlGyWJ+5g3WZW1ifuUG84QovKqW6+enN8AZB78pqwYpYUvMIBF3BBUq5gfWZO1iXuYX1mRuUUnsBdMt2ObItJ+8QERERESWi2M5UTURERJQubBARERFR6LFBRERERKGX0KDqvLw8lZ+fn6GiUDyWLVu2Syn1hzW+ksH6zK7Nmzdj165dhU4XGw/WZXalsy4B1me28drMHYnUZUINovz8fCxdyicqs0lEvk7XsVif2dWsWbO0HYt1mV3prEuA9ZltvDZzRyJ1yS4zIiIiCj02iIiIiCj02CAiIiKi0GODiIiIiEKPDSIiIiIKPTaIiIiIKPTYICIiIqLQY4OIiIiIQo8NIiIiIgo9NoiIiIgo9NggIiIiotBLaC0zoqJw5MgRKz1+/Hgdr1mzJup+n3zyiY7PPvtsK6906dI67tOnj5VXokQJHVesWDGxwlJWvf3221b6mWee0fHrr79u5bFuieJTq1atqHlbtmwpwpIULd4hIiIiotBjg4iIiIhCj11mVGT27dun4++++87Ke/LJJ3V8zTXXWHk9e/aM6/jnnHOOjkePHm3lVatWTceLFy+28latWqXjO++808q7++67dXz00UfHVQ7KrI8//ljH119/vZXXsWNHHZcpU6aoikSUU7Zu3arjmjVrZrEkRYt3iIiIiCj02CAiIiKi0GODiIiIiEKPY4gorfbv36/jtWvXWnmDBg3S8fTp06Me4+mnn7bSzZs317E5vmjTpk3WdmeccYaOzfFEANCwYUMdL1iwwMq79tprdeyOL9qxY4eOhw0bZuVxTFHRePfdd630c889p+Mff/zRyjvxxBN1zPpJvx9++MFKz5kzR8dDhgzR8dKlS6Me46677rLSQ4cOTVPpKBXmNBXm96f7vZesFi1a6Lhbt25WnvuZyBbeISIiIqLQY4OIiIiIQi8nuszc7pfJkyfr+MCBA1H3W79+vY4rVKhg5ZmPaT/yyCNWXu3atZMqZxiY3WTubNGxiIiOu3TpYuWZsw+bXSLJuuCCC6x0u3btdDxlyhQrb+bMmTr++9//buXVrVs35bJQZL/++quOx4wZY+WZs1ObM5ADQJMmTTJbsBAyuyXbtGlj5a1YsSLiPub1DABKKR1PnTrVynvggQd0XLly5aTLSYkxH60HgO7du+vY7DIzu7pSOb45XcZHH32U1DEzjXeIiIiIKPTYICIiIqLQY4OIiIiIQq/YjCEaPHiwlR44cKCOzUe9E9GoUSMdf/vtt1be/Pnzdfzmm29aeffee6+OBwwYkNS5w+7UU0+10hMnTtRx06ZNi7QsNWrU0LE7RskcP+FOI8AxROnjXsPmeC13zImpf//+Vvqqq65Ka7nCyJ2Wwvy+W758uZVnjhVq27atjh9//HFrO/P6fuONN6y8Q4cOJV9YStqVV14ZNa9Pnz4pH98dc1kc8A4RERERhR4bRERERBR6ge4ymzVrlo7NRzMB+3F6d1biRx99VMennHJK1ONXrVpVx+5tW7OrxJ1F84knntCx+TgpYHflhdFZZ52lY/e2eevWrXVcp04dKy+bj9vecMMNOv7yyy+tvIoVK+rYnd7h8ssvz2zBQmTu3LlWevTo0XHtF+v6pviZs7536tTJytu7d2/U/Xr37q1j83vx2GOPtbYzu9bca92cUdydfd79nqDUmL/LzMfgAXv26FjdafEyp78B/vh7Ooh4h4iIiIhCjw0iIiIiCj02iIiIiCj0AjWGyF1mwxyP4+Y9+OCDOnbH+JjjPuLlLgFQrlw5Hb/88stWnvmY9qJFixI+Vy4rU6aMju+5554sliR+Zn2anysAOP/883XcsGHDIitT2Nx+++1xb2s+Bn7FFVdkojg5z1213hw/smfPHivPvAYWLlyY1PnOPPNMHbtjS8xxhy1btrTyxo4dm9T5yOMukTF8+HAd16xZ08obOnRoyucbNmxY1Dx3hfsg4h0iIiIiCj02iIiIiCj0AtVl5s5suXjxYh27M9AW5QzRo0aNstLmbMYzZswosnJQZhw8eFDH7urd7gzmlBpzeounn35axzt27Ii6z7XXXmulzdmpS5Uqlb7Chch9991npc3PvbtS/XHHHafjffv2WXnmI/NfffWVjt1Z3adMmaLjSZMmWXmHDx/Wcfny5QstO8XPXMHeNWTIECvtdqElw+0ONblDW4KId4iIiIgo9NggIiIiotBjg4iIiIhCL1BjiGbPnm2lzf5p93HoTDNXu7/77rutvGbNmunYLCMVT+YSI/369Yu6Xfv27YuiODntm2++0XGsFbXNpR9uuukmK4/XXOrcVetjmTZtmo7NR+QBe8yPuezNb7/9Zm1nLnHkjlEymeOQKDnmo/Zbt2618sxxQulYnsNlLgdSHJbqcPEOEREREYUeG0REREQUeoHqMnNVqFBBx6effnpGz+Wusnz11Vfr2F3R3p1NlYqXnTt3Wunnnnsurv0aNGiQieKESrRHb82pLACgTZs2Or7wwgszWqYwclecj5fbpRWr+ysZffv2TevxwijW4+3ulAepijUzdawu8aDiHSIiIiIKPTaIiIiIKPQC3WW2d+9eHc+ZM8fKu+SSS1I+/rhx43TsztxqzpzrdtcNGjQo5XNT9nz22WdWetu2bVkqSe6bPn26lY62GHLz5s2tNBf1zKxXXnnFSt9yyy06/uKLL6y89evX67hJkyZWnrmQdpcuXXTcuHFjaztzgdhq1apZeeZs8O7TaZQ480kvV4sWLdJ6rhEjRkTNS+QpNrPrLZszWvMOEREREYUeG0REREQUemwQERERUegFagxR1apVrbTZd92hQwcrr127djquXr16XMd/9913rfSWLVt0fOTIkaj7derUyUqXK1curvNR/FatWqVjd8X5eHXu3FnHlSpVirrd1KlT4z6m2Z+djtWgw6ZHjx5W2hwXaHKvb8qsk046yUq/9dZbOv7555+tvP379+vYva5KlSql46OOiv739YYNGyIeD7Af3W/dunWsYlMAxJoJO9bs1OZ+7jghcxwnxxARERERZREbRERERBR6geoyMxcRBICePXvq+O2337by3HQ83C65O+64Q8fu48EbN27Ucbdu3RI+F/3Rq6++qmN36gJzmgP3lnq8zEV4S5QoYeWZdf3SSy9FPYb5eDAAPPbYYzo2uwcoOrOr+/Dhw1aeuTCr+ajtjTfemPmCUVzMBVsjpZOxefNmHbtdcpReZte+26Vldk+6v9fM7q4aNWpEPf6UKVOi5pmP/Ccyi3m6Z9BOFu8QERERUeixQUREREShxwYRERERhV6gxhC5j3S+8cYbOn7vvfei7vfhhx9a6by8PB2bq2abPwfsFbbXrFlj5ZljiOrXrx+r2GT45ptvdOyuUr5p06ao+5lLAMRaGuXss8/Wcb169aw8s9/b/UxcfPHFOo41xYI7TojjhhI3ZswYHbvjRfLz83V88803F1WRiELDnE7GfYR9+PDhOp48ebKV56aTYY5Dcqcp6dq1q44TWdajKPEOEREREYUeG0REREQUeoHqMnOVLPl78Tp27Bh1u1h56WDOsgoAZ511VkbPV5xt375dx7G6yNzutNmzZ+s42W4q83H9Rx55xMqL1U1Gqdm9e7eVXrJkSdRt27Ztm+niUACNHz9ex0qpLJYkXMypLdz066+/HnU/89F681F6wH6U3310P9YxiwPeISIiIqLQY4OIiIiIQo8NIiIiIgq9QI8hCoply5ZZaY4h+t3q1autdL9+/aJua06r8K9//cvKS2bc0LvvvmulBw4cqGO339usM3dKeXNZgbVr11p5X375pY7r1q2bcBnDYMiQIVZ6/vz5Ubdt0KBBpotDAWRec+71V7lyZR03adKkyMoUdrEefTfzYi3BMXTo0LSWKdt4h4iIiIhCjw0iIiIiCj12mVHC9u3bp+MOHTpYeeZM1a6XX35ZxxdddFFS554xY4aOe/ToYeX99NNPOu7SpYuVZz5GetRR9t8BDz30kI7NmVwB+5FydplF1rNnTyv96KOPRt3WvMV+6623ZqxMVHwcPHhQxz/88IOVV6VKlaIuDiXAnY26uOMdIiIiIgo9NoiIiIgo9NggIiIiotDjGKI4HH/88dkuQqCYff6xxgy5ateuHTVv3bp1Ol6zZo2VZ44bevPNN3VsjhkC7Gnjzz77bCvPHTdkeuCBB6LuZz4STETJibWMj/l9Yi6/Q8HkLteRS3iHiIiIiEKPDSIiIiIKPXaZxaF169bZLkKgVKxYUccPP/ywlXf//fdH3e/ee+/Vsdt9NmnSJB3Xr1/fyrvkkkt0/NRTT+m4RYsW1nbmI6ClS5eOWo5YLr300qT2C7P8/Hwrbc5W7n4+zMeqp0+fruOOHTtmpnAUOO5q97/99puO3cfuKXjM1e7NGCj+j+HzDhERERGFHhtEREREFHpsEBEREVHohXoMkfnY9oIFC6y80047TcfJrMSey0qUKKFjc1wQADRu3FjHX331lZU3YMAAHdepUyfqfubYEsB+ZL5MmTJJlJgyya2TgQMH6tidGsGcxoDjhsKjadOmOl68eHEWS0KJOuecc6z0tm3bslSSzOMdIiIiIgo9NoiIiIgo9ELdZfb555/r2Hz0E7C7zEqWDPXbFJPZfQYAl112WdRtb7nllkwXhwJm5MiR2S4CBYD5ferKy8vT8cknn1wUxaEEfPTRR9kuQpHhHSIiIiIKPTaIiIiIKPTYICIiIqLQC/XgmJUrV0bNizUWhoiI4meukP7iiy9aeQ899JCOTz311CIrE5GLd4iIiIgo9NggIiIiotALdZeZqV69ela6V69eWSoJEVFuqVKlio6XLFmSxZIQRcc7RERERBR6bBARERFR6LFBRERERKEX6jFEt956a8SYiIiIwoV3iIiIiCj02CAiIiKi0BOlVPwbi+wE8HXmikNxOFkpVTUdB2J9Zh3rMnekrS4B1mcA8NrMHXHXZUINIiIiIqJcxC4zIiIiCj02iIiIiCj02CAiIiKi0GODiIiIiEKPDSIiIiIKPTaIiIiIKPTYICIiIqLQY4OIiIiIQo8NIiIiIgq9/w/oVbttJ2zBcQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x864 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 定义比对可视化函数\n",
    "def plot_images_labels_prediction(images,  # 图像列表\n",
    "                                  labels,  # 标签列表\n",
    "                                  prediction,  # 预测值列表\n",
    "                                  index,  # 开始显示的索引\n",
    "                                  num=5):  # 缺省一次显示5张\n",
    "    fig = plt.gcf()  # 获取当前图表，get current figure\n",
    "    fig.set_size_inches(10, 12)  # 1英寸等于2.54cm\n",
    "\n",
    "    if num > 25:  # 最多显示25张图片\n",
    "        num = 25\n",
    "\n",
    "    for i in range(0, num):\n",
    "        ax = plt.subplot(5, 5, i + 1)  # 获取当前要处理的图片\n",
    "        ax.imshow(np.reshape(images[index], (28, 28)), cmap='binary')  # 显示第index个图像\n",
    "        title = 'label = ' + str(np.argmax(labels[index]))  # 显示标签的标题\n",
    "        if len(prediction) > 0:  # 如果有预测结果的话，添加显示预测的标题\n",
    "            title += ',predict = ' + str(prediction[index])\n",
    "        ax.set_title(title, fontsize=10)  # 显示图上的标题\n",
    "        # 不显示坐标轴\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "        index += 1\n",
    "    plt.show()\n",
    "\n",
    "\n",
    "# 可视化第100-109张测试图片的预测结果对比\n",
    "plot_images_labels_prediction(mnist.test.images,\n",
    "                              mnist.test.labels,\n",
    "                              prediction_result,\n",
    "                              100, 10)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
